Large-scale automated acoustic monitoring of birds and the challenges of field data

Modern technologies for the automated acoustic monitoring of animal communities enable species surveys that yield data in unprecedented volumes. Interpretation of these data bring new challenges related to the need of automated species identification. Coupling automated audio recording with automated species identification has enormous potential for biodiversity assessment studies, but it has posed many challenges to the effective use of techniques in real-world situations. This thesis develops new methods in the field of bioacoustics applied to automated monitoring of vocal species in terrestrial environments. Specifically, I developed automated methods to classify acoustic ecological data generated under the two most common contexts used in ecology: identification of vocalization data stored in acoustic libraries of sounds and identification of vocalizations in audio data collected from the field, through e.g., acoustic monitoring programs. The methods bring key developments across the entire pipeline for automated acoustical identification, connecting techniques from the data acquisition in the field to the ecological modelling of data identified utilizing automated classification methods. I show the performance of methods over huge datasets, compare them with alternative cutting-edge techniques and provide an ample study case of Amazonian bird communities to show the tools in practice. The methods in this thesis are available as open source and ready-to-use software capable to work directly on field data collected from acoustic monitoring efforts.Nykyaikaiset/modernit teknologiat/tekniikat eläinyhteiskuntien automattiseen akustiseen monitorointiin mahdollistavat lajitutkimuksen, joka tuottaa ennennäkemättömän määrän tutkimusaineistoa. Tällaisen tutkimusaineiston tulkinta aiheuttaa uusia haasteita (kuten) tarpeen automatisoituun lajitunnistukseen. Automatisoitu audiotallennus yhdistettynä automaattiseen lajitunnistukseen luo uusia mahdollisuuksia biodiversiteetin inventointiin/ luontotyyppien seurantatutkimukseen, mutta ne ovat myös aiheuttaneet monia haasteita menetelmän käyttämiseen todellisissa tilanteissa. Tämä tutkimus kehittää uusia menetelmiä maalla elävien ääntelevien lajien automaattiseen seurantaan bioakustiikan tutkimuksen kentälle. Kehitin ennenkaikkea automatisoituja menetelmiä kahden tyypillisimmän akustisessa ekologiassa käytetyn aineiston; lajiäänitteiden tietokantojen sekä lajiäänitteiden maastoaineiston tallenteiden, luokitteluun. Nämä menetelmät kehittävät olennaisesti koko automatisoidun akustisen tunnistuksen kenttää yhdistäen maastoaineiston automatisoidun keruun automaattisten luokitusmenetelmien avulla tunnistettujen tietojen ekologiseen mallintamiseen. Osoitan menetelmien toimivuuden (käytännössä) erittäin suurten aineistojen avulla vertaillen niitä tämänhetkisiin huipputekniikoihin sekä tarjoan laajan Amazonin lintuyhdyskuntia koskevan tapaustutkimuksen/tutkimusesimerkin osoittaen näin välineiden/menetelmien toimivuuden käytännössä. Tutkimuksessa tuotetut menetelmät ovat saatavilla avoimen lähdekoodin sekä käyttöönotettavan/toimivan ohjelmiston muodossa maastoaineiston käsittelyä varten

Uso de clareiras por aves na Amazônia Central: uma abordagem quantitativa considerando detecção imperfeita

The vastness of Amazon forests creates a false appearance of homogeneity; however, forest organisms do not occur with the same probability in every location. Tree-fall gaps, in dynamic succession from tree fall to complete gap disappearance, create spatial variation in vegetation density and are a strong source of habitat heterogeneity inside the forest. Aiming to understand effects of habitat heterogeneity on habitat use and local distribution of bird species, we investigated differences in site-occupancy by birds in closed forest and in the vicinity of tree-fall gaps. We tested predictions of different occupancy for 68 bird species of terra firme forest of the Amazon Guianan shield, north of Manaus, Brazil. Specifically, we tested how occupancy of each habitat type should vary among three pre-determined species categories of habitat use. We used autonomous recording devices to collect data in 57 closed forest and 50 tree fall gap sites from June to October 2010. Our analytic approach, a multi-species hierarchical community model that accounts for the possibility of detection failure, leads to strong inferences about changes in occupancy between habitat types, species and groups of species. In the broadest sense, our results did not confirm predicted differences in occupancy across groups. In general hypothetical gap-seeking , gap-avoiding and neutral groups had similar occupancy probabilities within the same habitat; only one group had different occupancy between habitat types. At the species-level, only 25% of investigated species conformed to predicted occupancy differences between habitat types. Hypothetical gap-seekers had the highest detection probabilities in both habitats. Interestingly, hypothetical gapavoiders had higher probability of detection on gap areas than in closed-forest areas. Neutral species, those predicted to occupy any habitat type with the same probability, had the highest detection probabilities in closed forest across groups. In general, the ornithological classification of bird habitat preferences used to form our occupancy predictions did not match our field estimates. These results highlight the need to test widely accepted classifications of habitat preference as well as the usefulness of hierarchical community models for testing hypotheses about groups of species.A imensidão da floresta Amazônica cria uma falsa aparência de homogeneidade; no entanto, os organismos da floresta não ocorrem com a mesma probabilidade em todos os locais. Clareiras, em dinâmica de sucessão desde a caída da árvore até o desaparecimento completo da abertura no dossel, criam variação espacial na densidade da vegetação e são importante fonte de heterogeneidade de habitats no interior da floresta. Com o objetivo de entender os efeitos da heterogeneidade de habitat no uso do habitat e na distribuição local de espécies de aves, nós investigamos diferenças de ocupação por aves em áreas de floresta fechada e de clareiras. Testamos previsões sobre ocupação diferencial para 61 espécies de aves de floresta de terra firme na Amazônia Central, norte de Manaus, Brasil. Especificamente, testamos como a ocupação de cada tipo de habitat deve variar entre três categorias pré-determinadas de uso de habitat pelas espécies. Utilizamos gravadores autônomos para coletar dados em 57 pontos de floresta fechada e 50 pontos de clareiras no período de Junho a Outubro de 2010. Nosso contexto analítico, um modelo hierarquico de comunidade que considera a possibilidade de haver falhas na detecção, permitiu inferências robustas sobre diferenças de ocupação entre tipos de habitat, espécies e grupos de espécies. Num senso amplo, nossos resultados não confirmaram as previsões sobre diferenças de ocupação entre os grupos. Em geral, os grupos hipotéticos favorecem clareiras , evitam clareiras e neutro apresentaram probabilidade de ocupação similar dentro do mesmo habitat; apenas um grupo teve ocupação diferente entre os tipos de habitat. No nível específico, apenas 25% das espécies investigadas corroboraram nossas previsões sobre diferenças de ocupação entre habitats. O grupo hipotético que favorece clareiras teve as maiores probabilidades de detecção em ambos os habitats. Curiosamente, o grupo hipotético que evita clareiras teve maiores probabilidades de detecção em áreas de clareira do que em floresta fechada. Espécies neutras , previstas para ocupar qualquer tipo de habitat com a mesma probabilidade, tiveram as maiores probabilidades de detecção em floresta fechada dentre os grupos. De modo geral, as classificações ornitológicas de preferência de habitat utilizadas para formar nossas previsões sobre ocupação não corresponderam a nossas estimativas de campo. Tais resultados ressaltam a necessidade de se testar essas classificações amplamente aceitas de preferência por habitat. Os resultados ressaltam ainda a utilidade de modelos hierárquicos de comunidade para testar hipóteses sobre grupos de espécies

PROTAX-Sound : A probabilistic framework for automated animal sound identification

Autonomous audio recording is stimulating new field in bioacoustics, with a great promise for conducting cost-effective species surveys. One major current challenge is the lack of reliable classifiers capable of multi-species identification. We present PROTAX-Sound, a statistical framework to perform probabilistic classification of animal sounds. PROTAX-Sound is based on a multinomial regression model, and it can utilize as predictors any kind of sound features or classifications produced by other existing algorithms. PROTAX-Sound combines audio and image processing techniques to scan environmental audio files. It identifies regions of interest (a segment of the audio file that contains a vocalization to be classified), extracts acoustic features from them and compares with samples in a reference database. The output of PROTAX-Sound is the probabilistic classification of each vocalization, including the possibility that it represents species not present in the reference database. We demonstrate the performance of PROTAX-Sound by classifying audio from a species-rich case study of tropical birds. The best performing classifier achieved 68% classification accuracy for 200 bird species. PROTAX-Sound improves the classification power of current techniques by combining information from multiple classifiers in a manner that yields calibrated classification probabilities.Peer reviewe

Animal Sound Identifier (ASI) : software for automated identification of vocal animals

Automated audio recording offers a powerful tool for acoustic monitoring schemes of bird, bat, frog and other vocal organisms, but the lack of automated species identification methods has made it difficult to fully utilise such data. We developed Animal Sound Identifier (ASI), a MATLAB software that performs probabilistic classification of species occurrences from field recordings. Unlike most previous approaches, ASI locates training data directly from the field recordings and thus avoids the need of pre-defined reference libraries. We apply ASI to a case study on Amazonian birds, in which we classify the vocalisations of 14 species in 194504 one-minute audio segments using in total two weeks of expert time to construct, parameterise, and validate the classification models. We compare the classification performance of ASI (with training templates extracted automatically from field data) to that of monitoR (with training templates extracted manually from the Xeno-Canto database), the results showing ASI to have substantially higher recall and precision rates.Peer reviewe

PROTAX-Sound: A probabilistic framework for automated animal sound identification

Autonomous audio recording is stimulating new field in bioacoustics, with a great promise for conducting cost-effective species surveys. One major current challenge is the lack of reliable classifiers capable of multi-species identification. We present PROTAX-Sound, a statistical framework to perform probabilistic classification of animal sounds. PROTAX-Sound is based on a multinomial regression model, and it can utilize as predictors any kind of sound features or classifications produced by other existing algorithms. PROTAX-Sound combines audio and image processing techniques to scan environmental audio files. It identifies regions of interest (a segment of the audio file that contains a vocalization to be classified), extracts acoustic features from them and compares with samples in a reference database. The output of PROTAX-Sound is the probabilistic classification of each vocalization, including the possibility that it represents species not present in the reference database. We demonstrate the performance of PROTAX-Sound by classifying audio from a species-rich case study of tropical birds. The best performing classifier achieved 68% classification accuracy for 200 bird species. PROTAX-Sound improves the classification power of current techniques by combining information from multiple classifiers in a manner that yields calibrated classification probabilities

Gain in computational and statistical efficiency due to feature pre-selection.

<p>The classification accuracy for the Random Forest algorithm (y-axis; black dots) was calculated over the training data and for different amounts of cross-correlation features (x-axis; grey bars). The choice of the quantity of features to be used as PROTAX-Sound predictors was based on the configuration which showed the highest classification accuracy based on as few features as possible (dashed blue line).</p

A schematic overview of PROTAX-Sound, a probabilistic classification system for animal sounds.

<p>Input files consist of labeled reference audio and field audio to be classified. The final outputs are the predicted classification probabilities for segments of field audio. Green boxes represent PROTAX-Sound functions; white boxes are inputs and outputs of these functions. The acoustic features and PROTAX-Sound predictors are calculated in the same way for both reference and query samples. The distances calculated from the MFCC features are used as PROTAX-Sound predictors. The cross-correlation features are used as input in the random forest model, the output of which is used to calculate PROTAX-Sound predictors. Mel-scaled log-power spectra of selected frames are used as input in the convolutional neural network, the output of which is used to calculate PROTAX-Sound predictors in the same way as for random forest. Panel a) shows the overall framework and panel b) the feature extraction pipeline (box 2 in panel a) in more detail, illustrated with MFCC features, cross-correlations features classified by Random Forest and power spectra features classified by convolutional neural network.</p

Accuracy and bias of 1766 test samples identified by different versions of PROTAX-Sound, Random Forest and Convolutional Neural Network classifiers.

<p>Panel a) shows reliability diagrams for the best outcome species (x-axis) and the cumulative correctness of the prediction (y-axis). The six lines correspond to the raw output of Random Forest (RF), the raw output of Convolutional Neural Network (CN) and the PROTAX-Sound models that use MFCC, RF, CN or their combination as predictors. The model-predicted probabilities are calibrated if the lines follow the identity line (the grey diagonal line), and they are the more accurate the higher the lines reach. Panel b) shows the distribution of p-values for the 200 species classified by PROTAX-Sound (MFCC+RF+CN), asking if the classifications are not calibrated for some particular species. Panel c) shows the distribution of the highest PROTAX-Sound (MFCC+RF+CN) probabilities predicted for each of the test samples. Panel d) shows the highest PROTAX-Sound (MFCC+RF+CN) probability against the number of reference samples. In this panel, each dot corresponds to each of the 200 species, and the probabilities are averaged over all test samples that belong to the species.</p

Illustration of the performance of PROTAX-Sound (MFCC+RF+CN) for selected example species.

<p>Each panel corresponds to one focal species, and each bar corresponds to a single query sample originating from that focal species. The colors summarize the predicted probability distribution of species identity over all species in the reference database, presented in descending order of probabilities. The five highest probabilities are shown in distinct colors, whereas the remaining probabilities are summed together and shown by the yellow bar. The predicted label is the species that PROTAX-Sound assigned the highest probability (the dark blue part of the bar), and the number after labels is the rank of the identification that corresponds to the true species. For each focal species is also shown the proportion of correct identifications (PCI; fraction of cases where the identity with highest probability corresponds to the true species) and the mean highest probability assigned by PROTAX-Sound (MHP; average over the highest probabilities, whether they represent the true species or not). The species have been selected to show contrasting cases of identification uncertainty: a) <i>Vanellus chilensis</i>; b) <i>Crypturellus soui</i>; c) <i>Batara cinerea</i>; d) <i>Henicorhina leucophrys</i>. For full names of all species and their abbreviations, see supporting information <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0184048#pone.0184048.s003" target="_blank">S2 Table</a>.</p